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A!?ST!?ACT 

This is an evalua*-icn cf a program designed to 
imprcve the reading skills of Title T eligible children in grades u-6 
through the integration of a total art prooram' with a total reading 
proaram. A total of 1,176 New York City school children were served 
by the prograir, which was conducted bv the Board of Education in 
association with five cfty museums. Presen^-ed in this evaluatioi; are 
a brief description of proaram activities and achievement objectives, 
as well as the results of the California Achievement Test (pre and 
post) and the Wisconsin Desian Skill Development Test, both of which 
were administered to participatina students. Based on these results 
i* is concluded that the Title I "Learning to Bead Through the Arts" 
l^rdect was a success. Fecommendaticns are offered regarding: (l) the 
expansion of the program within New York City end as a nationwide 
resource center: (2) expansion of the student population served to 
include special education students, bilinaual students, and students 
vounqer than 'ith grade; d) expansion cf evaluation procedures to 
focus on other than readina skills; and (U) the continued allocation 
of funds for the development of trair.ino materials. (fiC) 
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1. 

*Titlt I Childrtn's Progrim: Learning to Read Through the Arts 
The Boerd of Education of the City of New York 

Developer/Demonstrator Project, National Diffusion Newtvork 
I United States Office of Education 



Bemadette C. O'Brien 
Citywide Coordinator 



PROGRAM DESCRIPTION 



In school year 1978-1979, the Board of Education of the 
City of New York offered two twenty-eight-week program cycles from 
October 1978 -through June 1979. The program was associated with 
the Queens Museton, Staten Island Children's lluseum, The Bronx 
Museum of the Arts, New York Aquarium, and the Brooklyn Museum, 
iniis Title I ESEA program was scheduled to service a total of 
1,176 children (grades 4-6) performing at least one year below 
grade level In reading. In the five boroughs in New York City, 
Manhattan, Staten Island, Queens, Brooklyn and the Bronx. The 
program serviced 250 Title I eligible children (grades 4-6) at 
each site except Staten Island which serviced 176 children. 
One hundred and twenty-five (125) children participated in the 
program on Mondays and Tuesdays from October 16, 1978 through 
June 5, 1979 and a different 125 children participated on 
Wednesdays and Thursdays, October 18, 1978 through June 6, 1979, 
at each site. In Staten Island 88 children participated on 
Mondays and Tuesdays and a different 88 participated on Wednesdays 
and Thursdays. The program took place from 12:30 p.m. to 4:30 p.m. 
at the Manhattan, Queens, Brooklyn and Bronx sites, and from 
11:30 a.m. to 3:30 p.m. at the Staten Island site. 

The Title I Children's Program: Learning to Read Through 
the Arts is an intcmsive. Individualized reading program that 
focuses on the improvement of reading skills through the integration 
of a total art program with a total reading program. Listening, 
speaking, writing and reading techniques are stressed in the reading - 
oriented arts workshops. A diagnostic, prescriptive approach to 
reading is stressed in the reading workshops. The program involves 
study of reading-workshops as well as reading-oriented arts 
workshops of the students' choice, including: Dance, Theater, 
Music, Painting, Mixed Media, Sculpture, Work on Paper, Graphics, 
Printmaklng, Crafts, and Super- 8 Film and Photography. 

During the twenty- eight -week program each student participated 
in two reading-oriented arts workshops. The students received 
reading instruction in the reading-oriented arts workshops through 
a listening, speaking, reading, writing approach to reading for 
five hours of the program each week. Each participating student 
also met with a reading teacher in small groups, or individually, 
at least three hours of the total eight workshop -hours per week 
for the twenty-eight weeks of the program. 



5 



2. 



One day of each month was set aside for field trips to 
museums, galleries, artists ' studios, arts and educational 
resources center and libraries. In addition, trips were made 
to attend theatre, dance, music, and film performances outside 
the associated museums' facilities. Some of these field trip 
days were set aside for special events which included live 
theatre, dance and music performances inside the associated 
institutions: The Queens Museum, Staten Island Children's 
Museum, The Bronx Museum of the Arts, New York Aquarium and the 
Brooklyn Museum. Occasional afternoons were also devoted to 
film programs. The program entailed the use of the museum's 
facilities and galleries. 

All children attending the program were given introductions 
to the associated museums! collections and special exhibitions. 



SPECIAL DAYS: 



At the end of the program the New York City Board of Education 
and the associated institutions invited parents to attend Performing 
Arts and Film Festivals at the respective sites. During the pro- 
gram, the "Learning to Read Through the Arts" Exhibition was presented 
which included examples of the children's work and pUotographs of 
the program in action. 



PARENTS* WORKSHOPS: / 

Up to 30 parents at each site participated in a series of 
parents^ workshops for one and a half hours per week for ten weeks 
for each program on how children learn and family life education. 
These workshops were led by the social worker and assistant 
coordinator for the program. 



« 



ERIC 



6 



3. 

4 • 



Evaluation Objectives and Criterion of Success 

Nornv Referenced Test 

On tlie California Achievement Test Form C (Levels 13-16) pre and post-test 
student Normal Curve Equivalent scores will reflect a significant Treatment Effect 
(0) . Data will be analyzed using United States Office of EducaUon iAqdel Al . 

Criterion-Referenced Test 

Seventy percent of the Title I Children' s Program: Learning to Read Through 
the Arts students will demonstrate mastery of at least 5 slciils on the Wisconsin 
Design Skill Development Test (Levels B/ C and D, forms P and Q) which they 
had not mastered when they entered the program. 
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■ QARO or COUCATIDN OF THE CITY OF NEW YORK 

OFFICE OF EDUCATIONAL EVALUATION 
UQ LIVINGSTON STRCCT. BROOKLYN. N. Y. UZOI 



Alan & Bcumnim. Pm.Ol 

ACTINO OHVKCTOM 



SUMMARY OF RESULTS 
Title I Children's Program: Learning to Read Through the Arts. The Board 
of Education of the City of New York, Developer/ Demonstrator Project. 
PPnfiPAM Ni.tinr^a1 Diffus ion Network. United States Office of Education 

PROGRAM # 5QQ9 PROJECT I 9fi722 - 98726 



EVALUATION INSTRUMENT: Caifornia Achievement Test. Form C, Leve ls 13-16 

TYPE OF INSTRUMENT: CRT ^RT X . OMG. 

TEST HATgS: October. 1978 / May-June, 1979 _ . 



NUMBER OF STUDENTS IN PROGRAM: 1.176 



GRADES LEVELS OF ^^TUDENTS : grades and /or 9-12 years old.' 
NUMBER OF SITES: 5 



DATA ANALYSIS PROCEDURES: 

Pre/ post test data compared using USOE Model Al via NCES. 



SUMMARY OF RESULTS: 

Treatment Effect « 18.41 

4th grade - Treatment Effect ■ 26.63 
5th grade - Treatment Effect « 14.38 
6th grade - Treatment Effect « 13,11 

WERE THERE OTHER EVALUATION I NSTRUMENT ( S ) USED IN PROGRAM? Yes_X. 



Wisconsin Design Skill Development Test CRT X NRT OMG. 

CRT NRT OMG. 



gj^^herc were a few third graders Included in the 9^2 year old group. 



No 
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Report of Evaluation Results-Norm Referenced Test 19 78-1979 
Students in the Title I Children's Program; Learning to Read Through the Arts, 
The Board of Education of the City of New York, Developer/Demonstrator Project, 
National Diffusion Network, United States Office of Education were administered 
California Achievement Test Pre-tests and Post-tests {Form C, Levels 13-16), The 
pre-test/post-test djjta has been analyzed using the USOE Model Al, via NCE 
(Normal Curve Equivalent) scores in accordance with the evaluation design. 

Students in the program were enrolled in grades 4-6 and/ or were 
9-12 years old. The vast majority of the students were in the fourth to sixth grades. 
Complete pre-test/post-test data was obtained for 882 of the 1X76 students in the 
program. The data and results reported reflect the pre-test/post-test NCE scores 
for: 

3 (out of 3) third-graders , 
308 fourth-graders, 
370 fifth-graders and 
201 sixth-graders 

The results are presented as pre-test/post-tsst NCE means for each grade, 
as well as for the entire program. 

It should be noted that Normal Curve Equivalent scores are based on an equal 
intarval scale, with a mean of 50. Unlike percentiles, NCE scores can be aggregated 
and averaged. According to USOE Model Al, it is expected that, without treatment, 
the NCE score a student attains on the post- test will remain the same as the score 
attained on the pre-test. That is to say, under no-treatment conditions, a student 
is expected to remain In the same position, relative to other students, on both the 
pre and post-tests. 

A program's Impact, or treatment effect. Is measured by comparing the mean 

ERIC 
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observed post- treatment performance of a group of students (actual post-test 
mean NCE) with the expected no-treatment performance. It is assumed that with- 
out Title I treatment, the expected post-test NCE mean will be the same as the pre- 
test NCE mean. When the treatment effect is greater than zero (0) , the gain can be 
attributed to the effectiveness of the program. ( E5EA Title I Evaluation and Reporting 
System, Technical Paper Number 2. Interpreting NCE s. G. Kasten Tallmage, 1976) . 

The substantial increases in mean NCE scores for the entire program, and at 
each grade level, can be interpreted as evidence of the effectiveness of Title I 
Children's Program: Learning to Read Through the Arts in improving the reading levels 
of students in the program. 



California Achievement Test NCE Pre-test/lPost-test Means 

Number 

of Mean Pre-test Mean Post-test Treatment 
Group Represented Students NCE NCE Effect 

Ail students in Title I 882 26.47 44.88 18.41 

Children's Program; Learning 
to Read Through the Arts 
taking pre/post-test 

All third graders taking 3 37.67 51.67 14.00 

pre/post-test 

All fourth graders taking 308 24.49 51.12 26.63 

pre/post-test 

All fifth graders taking 370 29.30 43.68 14.38 

pre/post- test 

All sixth graders taking 201 28.89 42.00 13.11 

pre/post-test 
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7. 

Title I Children's Program: Learning to Read Through the Arts 

The Board of Education of the City of New York 
Developer/Demonstrator Program, National Diffusion Network 

California Achievement Test NCE Pretest/Posttest Means 



Group 



Number of 



Pretest 



Posttest 



Treatment 



All students In 
Title I Children's 
Proaram: Learning 
to Read Thrc iQh 
the Arts taking 
pre/posttest. 


882 


26.47 


44.88 


18.41 


fourth graders - 
scross all sites 


308 


24.49 


51.12 


26.63 


Fifth graders - 
across ail sites 


370 


29.30 


43.68 


14.38 


Sixth graders - 
across ail sites 


201 


28.89 


42.00 


13.11 


Third graders - 
across all sites 


3 


37.67 


51.67 


14.00 
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Title I Children's Program: Learning to Read Throucih the Arts 

The Board of Education of the City of New York 
Developer/Demonstrator Project, National Diffusion Network 

BernadetteC. O'Brien 
Cltywlde Coordinator 

Frequency Distribution of NCE Scores on Pretest/Posttests for the 
Title I Children's Program: Learning to Read Through the Arts • 



fill Sites 



Groups Represented 



\\{ students In the 
)rQgram taking pre/ 
)osttest 



'ourth graders 
(across all sites) 



Fifth graders 
(across all sites) 



Sixth graders 
(across all sites) 



rhlrd graders 
(across all sites) 



Number 
of 

Students 



862 
(100 %) 



308 
(100%) 



370 
(100%) 



201 
(100%) 



PRETEST DISTRIBUTION 
by quartlle 



408 


454 


20 


- 


49 


583 


214 


36 


(47%) 


(51%) 


(2%) 




(6%) 


(66%) 


(24%) 


(4%) 


179 


122 


7 




6 


153 


122 


27 


(58%) 


(40%) 


(2%) 




(2%) 


(50%) 


(40%) 


(8%) 


151 


211 


8 




27 


286 


53 


4 


(41%) 


(57%) 


(2%) 




(8%) 


(77%) 


(14%) 


(1%) 


78 


118 


S 




16 


143 


37 


5 


(39%) 


(59%) 






(8%) 


(70%) 


(19%) 


(2%) 




3 








I 


2 





POSTTEST DISTRIBUTION 
by quartlle 
1-25 26-50 51-70 76-99 



/ 
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■qaro ar eouCATiON or the city or new yqrk 
□rncE or educational evaluation 

no LIVINGSTON. STREET. BRaOKLYN. N. Y. nZOl 



Alan & 3LUMNin. PH.a 

ACTina OtAtCTOI* 



SUMMARY OF RESULTS 
Title I Children's Proyram: Learning to Read Through the Arts, The Board 
of Education of the City of New York. Developer/Demonstrator Project, 
PROGRAM National Difft |<iifin f^ftwork- United Statos Office of Education 

PROGRAM # 5009 , PROJECT # 98722 - 98726 

(Levels B, C and D) 

EVALUATION INSTRUMENT: Wisconsin Design Skill Develo pment Test (Forms P and Q) 

TYPE OF INSTRUMENT: CRT X NRT OMG 

TEST DATES: Onpoinq during program intervention period. 

NUMBER OF STUDENTS IN PROGRAM: 1.176 



GRADES LEVELS OF STUD£NT S:4-6 grades and/or 9- 12 vears old.* 
NUMBER OF SITES: 5 



DATA ANALYSIS PROCEDURES: 

Analysis of the percentage of students in the program mastering 5 or more skills 
which they had not mastered when they entered the program. 70S should master at leasv 
5 skills not mastered at entry. 

SUMMARY OF RESULTS: 

825 of the students mastered at leasts skills not mastered at entry. 
78% of the students mastered at least 6 skills not m^»«;tered at entry. 



WERE THERE OTHER EVALUATION I NSTRUMENT ( S ) USED IN PROGRAM? Yes_JL 



No 



California Achievement Test CRT NRT X OMG. 

CRT NRT OMG, 
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E^lc^re were a few third graders included in the 9-12 year old group. 



Reoor: of Evaluation Results - Criterion Referenced Test 1978-1979 
The Wiaconsin Design Skill Development Test (Form P, Levels B, C and D) was 
administered to students entering the program. This test was used to determine each 
student's mastery of specific reading skills (e.g., phonetic analysis of consonants; 
comprehension of main ideas) . Based on student performance on the Wisconsin Design 
Skill Development Test, reading teachers were able to diagnose a student's mastery or 
nonmastery of specific skills . When the reading teacher and student decided that the 
student was ready to take a mastery post-test (Form Q, Levels B, C and D) in a skill 
previously not mastered, the test was administered individually to the student. Wis- 
consin Design Skill Development mastery tests were given on an ongoing basis during 
the intervention period. 

The criterion of success for the program on the Wisconsin Design Skill Develop- 
•ment test was that, during the Intervention period, 70% of the students master at least 
5 skills they had not mastered when they entered the program. In fact, 82% of the 
students in Title I Children's Program; Learning to Read Through the Arts mastered 5 
skills they had not previously mastered. The program exceeded its objective on the cri- 
terion referenced test. Tha majority (78%) of students in the program demonstrated 
mastery of at least 6 new skills. 

Results: 82% of students mastered 5 or more new skills 

78% of students mastered 6 or more new skills 
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Discussion of Results 1978-1979 

Results of th2 norm and criterion-referenced tests indicate that Title I Children's 
Program: Learning to Read Through the Arts, The Board of Education of the City of New 
.York has been successful in meeting its objectives for 1978-1979. The Treatment Effect 
of 18.41 Normal Curve Equivalent (NCES) on the California Achievement Test, reflects 
Impressive gains made by students in the program. The mastering data on the Wis- 
consin Design Skill Development Test also indicates that students have exceeded pro- 
gram objectives. Eighty-two percent (82%) of the students in the program mastered five 
or more new skills and 78% of the students mastered 6 or more new skills. 

The NC distribution of students by quartile reflects a dramatic shift in NCE 
scores for students in the program, especially those whose pre-test NCE scores were 
In the lowest quartile. Forty-seven percent (47%) of the entering students had scores 
between \ and 25. On the post-test 6% of the students were in the 1-25 quartile. The 
impact of the program on the lowest scoring students is particularly impressive. 

The gains made by students in Title I Children* s Program: Learning to Read Through 
the Arts are sizable. At each grade level students surpassed the criterion for success 
set in th^? evaluation design. The gains made by students can be attributed tc their 
participation in Title I Children's Program: Learning to Read Through the Arts. 
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12. 

Recommendations 

The Title I Children's Program: Learning to Read Through the Arts has proven 
successful in improving the reading skills of Title I students. Based on the evaluation 
.results and conversations with the Citywide Coordinator and program staff: 

1) It is recommended that additional sites be added to service Title I Children 
In the 32 school districts In New York City. 

2) It Is recommended that the program serve as a Resource Center for the- 32 
school districts In New York City, as well as school districts In the state and nation. 
Plug-In and satellite programs should be developed in school districts, with the 
nationally validated exemplary Title I Children's Program: Learning to Read Through the 
Arts, The Board of Education of the City of New York, DeveloperA)emonstrator Project, 
National Diffusion Network providing training and technical assistance to help maintain 
the high quality of that delivery system. 

3) It is recommended that the program provide service to student populations 
beyond those presently served, specifically: 

Special Education students could benefit from the Intensive, individualized 
reading instruction combined with an opportunity for self-expression and communication 
through the arts; 

Bilingual students could benefit from the comprehensive approach to art, 
culture , language and reading ; 

Students younger than the 4th - 6th graders (and/or 9-12 year olds) 
presently served could benefit from early Intervention, possibly preventing the need for 
later remediation. 

17 
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« * 

4) It is r^.commended that other aspects of the program, besides reading sJcills, 1 
evaluated to determine what variables have contributed to the reading improvement of 
students. Among the variables to examine could be: 

Change in interest and knowledge about the arts . 

Change in attitude toward reading and other academic subjects. 

The relationship between parent participation in workshops and student 
performance. 

If Title I funding is not available for such further evaluation, other funding 
might be sought for this purpose. 

5) It is recommended that money continue to be allocated for developing further 
training materials. Print materials, and a videotape or film of workshops in progress 
would facilitate training and technical assistance. 
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